Aspect based recovery system and method

ABSTRACT

The present invention includes an aspect based recovery system and method. States of a computer system may be defined by aspects. During operation of the computer system, the current state of the computer system may be determined by examining the current aspects. The examination of current state of the computer may include determining whether the current state of the computer is in an expected state with expected aspects. If the state of the computer is not in the expected state, aspect based recovery may modify selected aspects of the computer system to place the computer system in a known state.

BACKGROUND

[0001] Automation of testing holds great promise for testing products inan efficient and reliable manner. Ideally, test routines wouldautomatically execute in an arbitrary sequence without the need forhuman supervision. However, the reality of test automation has fallenfar short of its promise. Automated testing searches for exceptionaloccurrences in the system, such as failures in the system. When previousautomated testing schemes encountered an exceptional circumstance, theywere unable to address it in an effective manner. Previous automatedsystems might simply halt, requiring manual intervention by an operator.The manual intervention may include a rebooting of the system. Ifrebooting is unsuccessful, the operator might then reinstall thesoftware to fix the problem. This dependence on operator intervention iscostly from both a time and a human perspective. The testing softwaremay sit idly for a considerable amount of time before an operator mayintervene to fix the problem. Further, the time required to reinstallthe software may be considerable. Other automated systems have attemptedto compensate for an exceptional occurrence by causing, in effect, arestart or a reboot of the testing system. All of the routines necessaryto execute the testing system are reloaded, thereby reinitializing thesystem. The reloading of the system software may enable the testautomation to recover after a fatal error. However, this solution isagain inadequate since rebooting is a brute force method which affectsall of the system, including those portions of the system which areoperating correctly. The time involved in reloading the system softwarecan be significant, especially with systems that have large files thatmust be reloaded into RAM. Further, rebooting the system may destroyvaluable information for diagnosing the exceptional occurrence.

[0002] What is needed is a better method and system for analyzing andrecovering from exceptional occurrences in a system.

SUMMARY

[0003] In a first aspect of the invention, a method for executing anautomatic recovery routine for a computer system is provided. The methodcomprises providing a software routine for placing the computer systemin an expected state defined by an expected aspect; executing thesoftware routine on the computer system after which the computer systemis in a current state defined by a current aspect; automaticallydetermining whether the current aspect matches the expected aspect;automatically determining whether the current state is the expectedstate based, at least in part, on automatically determining whether thecurrent aspect matches the expected aspect; and executing an automaticrecovery routine if the current state is not the expected state, theautomatic recovery routine modifying the current aspect.

[0004] In a second aspect of the invention, a method for executing arecovery routine for a computer system is provided. The method comprisesexecuting a software routine; automatically determining, after executingthe software routine, whether the computer system is in an unknownstate; and automatically changing from the unknown state to a knownstate of the computer system, the step of changing consisting ofautomatically modifying aspects of the computer system to place thecomputer system in the known state.

[0005] In a third aspect of the invention, a method for executing arecovery routine for a computer system is provided. The method comprisesproviding a software routine for placing the computer system in anexpected state defined by expected aspects; executing the softwareroutine on the computer system after which the computer system is in acurrent state defined by current aspects; automatically analyzing thecurrent aspects of the current state of the computer system with respectto the expected aspects of the expected state; automatically determiningthe current aspects which differ from the expected aspects and thecurrent aspects which match the expected aspects; and automaticallymodifying the state of the computer system by automatically modifyingthe current aspects which differ from the expected aspects and notmodifying the current aspects which match the expected aspects.

[0006] In a fourth aspect of the invention, a method for executing arecovery routine for a computer system is provided. The method comprisesproviding a software routine for placing the computer system in anexpected state defined by expected aspects; executing the softwareroutine on the computer system after which the computer system is in acurrent state defined by current aspects; automatically analyzing thecurrent aspects of the current state of the computer system with respectto the expected aspects of the expected state; automatically determiningthe current aspects which differ from the expected aspects and thecurrent aspects which match the expected aspects; and automaticallymodifying at least some of the current aspects which differ from theexpected aspects without restarting the computer system.

[0007] In a fifth aspect of the invention, a method for recordingaspects for a computer system is provided. The method comprisesproviding a software routine for placing the computer system in anexpected state defined by expected aspects; executing the softwareroutine on the computer system after which the computer system is in acurrent state defined by current aspects; automatically analyzing thecurrent aspects of the current state of the computer system with respectto the expected aspects of the expected state; automatically determiningthe current aspects which differ from the expected aspects and thecurrent aspects which match the expected aspects; and recording thecurrent aspects which differ from the expected aspects.

[0008] In a sixth aspect of the invention, an aspect based recoverysystem for an application software on a computer system is provided. Theaspect based recovery system comprises a computer system having storedthereon the application software; programming code on said computersystem for executing at least a portion of the application software;programming code on said computer system for determining whether thecomputer system is in an unknown state; and programming code on saidcomputer system for changing from the unknown state to a known state ofthe computer system, the programming code for modifying aspects of thecomputer system independently of one another to place the computersystem in the known state.

BRIEF DESCRIPTION OF THE DRAWINGS

[0009]FIG. 1 is a block diagram of a computer system that may be used toimplement a method and apparatus for aspect based recovery according toone aspect of invention.

[0010]FIG. 2 is a flow diagram of one embodiment of potential stateswhen testing an application.

[0011]FIG. 3 is a flow chart of operation of one embodiment the aspectbased recovery system and method.

[0012]FIG. 4 is a class diagram for objects coded using object orientedprogramming in an aspect based recovery system and method described inFIG. 3.

[0013]FIG. 5 is a flow diagram for the object oriented coding in FIG. 4with no abnormal behavior detected while executing a test

[0014]FIG. 6 is a flow diagram for the object oriented coding in FIG. 4with abnormal behavior and recovery during precondition.

[0015]FIG. 7 is a flow diagram for the object oriented coding in FIG. 4with abnormal behavior and recovery during postcondition.

[0016]FIGS. 8a-e are a table of aspect based recovery for a medicalimaging application.

DETAILED DESCRIPTION OF THE INVENTION

[0017] All products during their operational life will eventually enteran unknown state of operation. The products may be software-basedproducts, software and hardware products, or hardware-based products.For example, a computer executing a software program may encounter afatal error resulting in the computer becoming unresponsive and“freezing.” Typically, when a product becomes unresponsive, the producthas entered an unknown state of operation which was not contemplated.Recovery of the product involves exiting the unknown state of operationand returning to a known state of operation. This recovery process maybe as simple as rebooting the computer which has frozen or as complex asreloading the operating system and other programs of the computer.However, there are times when these recovery processes of rebooting orreloading are inadequate. Another recovery process is contemplated whichis quicker and more efficient.

[0018] The operational life of a product involves many phases. One phaseof the product is the operation of the product in the field. Anotherphase, as discussed in the background section, involves testing ofproducts or applications. One type of testing is automated testing.Automated testing is typically performed prior to release of the productinto the field. The products may be subject to a variety of tests, suchas regression testing, integration testing, or validation testing, withlittle or no human intervention. For example, regression testing isparamount to quality assurance of changes to an established product,such as new versions. In addition to testing the intended effects of thechanges, regression testing verifies that the changes did not produceunintended effects such as altering behavior of establishedfunctionality. For example, a change in one routine for the softwarecode of a product may have both intended and unintended consequences forthe overall product resulting in the product entering an unknown state.Automated testing allows investigation into these consequences. Theproducts tested by the testing automation protocols may include productsentirely composed of software, software/hardware products and productsentirely composed of hardware. Testing programs may be designed to testa product or application.

[0019] Software testing typically results in the application under testentering an unknown state or encountering an exceptional circumstance,such as a crashing or hanging of the system. These exceptionalcircumstances may be due to an error in the testing program or due to aflaw in the application under test. For example, the testing program maybe causing the error that lead to entrance into the unknown state.Alternatively, a change in the software may have resulted in anunintended consequence that lead to entrance into the unknown state.Regardless, the exceptional circumstance should be analyzed to determineits cause.

[0020] Turning to the drawings, wherein like reference numerals refer tolike elements, FIG. 1 shows a block diagram of a computer that may beused to implement a method and apparatus for aspect based recoveryaccording to one aspect of invention. FIG. 1 and the followingdiscussion are intended to provide a brief, general, description of asuitable computing environment in which the embodiments may beimplemented. In one application, aspect based recovery is applied toautomation of testing of products. The computer may be used to test avariety of products including a software product, which may be loadedinto the system memory 22 of the computer, a hardware product, or asoftware/hardware product. The hardware product may be resident inhardware 56 and communicate via application hardware interface 55.Alternatively, the hardware or hardware/software product may be residentin remote device 49 and communicate with computer 20 via local areanetwork 51 or wide area network 52.

[0021] Although not required, the embodiments are described in thegeneral context of computer-executable instructions, such as programmodules, being executed by a computer. Generally, program modulesinclude routines, programs, objects, components, data structures, etc.that perform particular tasks or implement particular abstract datatypes. Moreover, those skilled in the art will appreciate that theembodiments may be practiced with other computer system configurations,including hand-held devices, multiprocessor systems,microprocessor-based or programmable consumer electronics, network PCs,minicomputers, mainframe computers, and the like. The embodiments mayalso be practiced in distributed computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote memory storage devices.

[0022] With reference to FIG. 1, an exemplary system for implementingthe one embodiment includes a general purpose computing device in theform of a conventional computer 20, including a processing unit 21, asystem memory 22, and a system bus 23 that couples various systemcomponents including the system memory to the processing unit 21. Thesystem bus 23 may be any of several types of bus structures including amemory bus or memory controller, a peripheral bus, and a local bus usingany of a variety of bus architectures. The system memory includes readonly memory (ROM) 24 and random access memory (RAM) 25. A basicinput/output system 26 (BIOS) containing the basic routines that helpsto transfer information between elements within the computer 20, such asduring start-up, is stored in ROM 24. The computer 20 further includes ahard disk drive 27 for reading from and writing to a hard disk, notshown, a magnetic disk drive 28 for reading from or writing to aremovable magnetic disk 29, and an optical disk drive 30 for readingfrom or writing to a removable optical disk 31 such as a CD ROM or otheroptical media. A software application under test may be loaded into thesystem memory 22 via either the hard disk drive 27, magnetic disk drive28, or the optical disk drive 30. The hard disk drive 27, magnetic diskdrive 28, and optical disk drive 30 are connected to the system bus 23by a hard disk drive interface 32, a magnetic disk drive interface 33,and an optical drive interface 34, respectively. The drives and theirassociated computer-readable media provide nonvolatile storage ofcomputer readable instructions, data structures, program modules andother data for the computer 20. Although the exemplary environmentdescribed herein employs a hard disk, a removable magnetic disk 29 and aremovable optical disk 31, it should be appreciated by those skilled inthe art that other types of computer readable media which can store datathat is accessible by a computer, such as magnetic cassettes, flashmemory cards, digital video disks, Bernoulli cartridges, random accessmemories (RAMs), read only memories (ROM), and the like, may also beused in the exemplary operating environment.

[0023] A number of program modules may be stored on the hard disk,magnetic disk 29, optical disk 31, ROM 24 or RAM 25, including anoperating system 35, one or more application programs 36, other programmodules 37, and program data 38. The application programs 36 may includethe software for the application under test. Further, the other programmodules 37 may include the software for the test scripts, used toexecute the test automation. Moreover, the program data 38 may includedata necessary for the test scripts to execute the test automation, suchas data for simulating operation of the application under test.

[0024] A user may enter commands and information into the computer 20through input devices such as a keyboard 40 and pointing device 42. Forexample, the user may use the keyboard 40 in order execute the testscripts software to test the application under test. Other input devices(not shown) may include a microphone, joystick, game pad, satellitedish, scanner, or the like. These and other input devices are oftenconnected to the processing unit 21 through a serial port interface 46that is coupled to the system bus, but may be collected by otherinterfaces, such as a parallel port, game port or a universal serial bus(USB). A monitor 47 or other type of display device is also connected tothe system bus 23 via an interface, such as a video adapter 48. Inaddition to the monitor, computers typically include other peripheraloutput devices (not shown), such as speakers and printers.

[0025] The computer 20 may further include an application hardwareinterface 55. The application hardware interface 55 enables interfacingto hardware 56 with the application under test, on a hardware level. Asdiscussed above, the application under test may include a hardwareaspect. The application hardware interface provides the inputs andoutputs necessary for simulation during testing. For example, thehardware 56 for the application under test may include sensors orrequire sensory input. In this instance, the application hardwareinterface 55 may provide that input, and receive any output via sensors57 in order to simulate the actual environment of the application undertest. In addition, the hardware 56 for the application under test mayinclude motors 58. The inputs and outputs for the motors may betransmitted to the computer via the application hardware interface 55.In one embodiment, the hardware 56 are devices for x-ray, computedtomography, magnetic resonance, ultrasound or other imaging systems.

[0026] The computer 20 may operate in a networked environment usinglogical connections to one or more remote devices, such as a remotedevice 49. The remote device 49 may be another computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the computer 20. The logical connections depicted in FIG. 1 include alocal area network (LAN) 51 and a wide area network (WAN) 52. Inaddition, remote device 49 may comprise a device which combines hardwareand/or software functionality. The remote device may comprise a varietyof devices, such as medical devices including: computed radiographymachine; computed tomography (CT) machine; digital X-Ray machine;mammography machine; magnetic resonance machine; nuclear medicinemachine; positron emission tomography (PET) machine; radiation therapy(RT) machine; ultrasound machine; X-ray angiography machine; X-rayradiofluoroscopy (XRF) machine; urology machine; surgery machine; andintensive care units (such as bedside monitoring). The computer 20 maythus interact with remote devices. As one example, the application maybe executed on computer 20 and interact with a remote scanning device.The application may thus receive data from the remote scanning deviceand may also control the scanning device (such as by initiating orhalting scans of a patient). Such networking environments arecommonplace in offices, enterprise-wide computer networks, intranets andthe Internet.

[0027] When used in a LAN networking environment, the computer 20 isconnected to the local network 51 through a network interface or adapter53. When used in a WAN networking environment, the computer 20 typicallyincludes a modem 54 or other means for establishing communications overthe wide area network 52, such as the Internet. The modem 54, which maybe internal or external, is connected to the system bus 23 via theserial port interface 46. In a networked environment, program modulesdepicted relative to the computer 20, or portions thereof, may be storedin the remote memory storage device. It will be appreciated that thenetwork connections shown are exemplary and other means of establishinga communications link between the computers may be used.

[0028] In order to execute the testing software better, the states ofthe automation software are analyzed. FIG. 2 shows a flow diagram ofpotential states when testing an application. These states are describedmerely for illustrative purposes and are not exhaustive of the states ina software automation system. Further, the states are described in aspecific test automation context wherein a typical testing sequenceproceeds through three different stages: precondition, test script andpostcondition. Again, these stages of testing are merely forillustrative purposes. Fewer, more, or different stages of testing maybe used.

[0029] The precondition stage describes the condition(s) that should bemet before a specific test is run. As merely one example, if thespecific test is analyzing a magnetic resonance (MR) image for apatient, the precondition may be to load the magnetic resonance imagefor the patient into a memory location accessible by the testingautomation software. The test script stage is the execution of the test.In the magnetic resonance example, the test script stage is the analysisof the specific MR image previously loaded in the precondition stage.Finally, the postcondition stage describes the condition(s) that shouldbe performed prior to executing another testing sequence. In effect, thepostcondition stage is a “cleanup” stage for cleaning up the systemprior to executing the next test. In the magnetic resonance example, thepostcondition stage may over-write the memory location previouslywritten by the precondition stage. The postcondition stage may beunnecessary if, after running the test script, the system may runanother test (be in system default, as described below).

[0030] Table 1 describes the state transitions present in FIG. 2 TABLE 1State Transitions Transition From To Via Note System System Power ONInitializing of the system; loading Off Default of the system softwareinto RAM (Block 60) System Script Precondition Expected conditions forDefault Begin Success establishing test script successfully (Block(Block 62) established 60) System System Unforeseeable Expectedconditions for Default Unknown Application establishing test script fail(Block (Block 64) Behavior 60) Script Script End Test Script Test scripthas been successfully Begin (Block 66) Success executed (Block 62)Script Script End Expected Target state may differ slightly Begin (Block66) Error from test success transition, but (Block difference in statesmay be 62) undetectable. The expected error may include a known error ina software routine. For example, a particular version of software mayhave a known error. Script Script End Non-Fatal Test script has notcompleted Begin (Block 66) Error successfully, but has not crashed or(Block caused a hangup in the system. 62) The target state may differfrom test success transition. Causes of a non-fatal error may includeerror in the test script or error in the application under test ScriptSystem Unforeseeable This usually occurs when the Begin UnknownApplication application under test crashes or (Block (Block 64) Behaviorhangs up. Some examples of 62) crashing may include an unexpected dialog(in user interface testing), a missed timeout, etc. Script System Postcondition Postconditions successfully End Default Success executed.Specifically, the (Block (Block 60) condition(s) prior to executing 66)another testing sequence successfully performed. Script SystemUnforeseeable Postconditions not successfully End Unknown Applicationexecuted. Condition(s) prior to (Block (Block 64) Behavior executingnext testing sequence not 66) successfully performed. Possible causesmay include: Bad testcase design. The test script may have an error inthe code. Non-Fatal Error cause. Although the test scrip did not hang upafter the non-fatal error, the postcondition cleanup cannot compensatefor the non-fatal error. Expected error cause. The expected errortransition has lead to Script End; however, the postcondition cleanup isnot flexible enough to accommodate that cleanup of the effects of theexpected error as well.

[0031] As shown in FIG. 2, the state of the system includes a systemunknown state. The system unknown state may include a state that cannotbe reached after reboot. For simplicity of illustration, FIG. 2 showsonly one system unknown state which may be entered via unforeseeableapplication behavior in a variety of ways. The states in the system maycomprise more than one unknown state. For example, each unforeseeableapplication behavior may lead to a different unknown state. In addition,for simplicity of illustration, only one precondition-testscript-postcondition sequence is shown. Additional sequences may beanalyzed.

[0032] The states of testing automation may be described in a variety ofways. One way is to describe a specific state of the testing automationas being composed of a number of aspects. An aspect is a substate thatcan be individually described as a function. For example, a tangibleaspect may be the database content A_(db). Other examples of aspects mayinclude current user (e.g., current user logged in), current user rights(e.g., security access of the current user), user interface (UI) state(e.g., message boxes open, dialog windows), user configuration (e.g.,settings for a specific user or set of users), site configuration (e.g.,configuration for a specific workstation), etc. The example describedbelow details some examples of these aspects. Summed up, all aspectsyield the State equation:

M=A _(db) +A _(UserConfig) +A _(SiteConfig) +A _(CurrentUser) +A_(UserPrivileges)+

[0033] The state, M, is described as a sum of independent aspect statefunctions as shown in the following:

M=M(db, UserConfig, SiteConfig, CurrentUser, UserPrivileges, . . . )

[0034] In one embodiment, the aspects may be described as functions. Aconcrete value of this function may be defined. For example, in thedatabase aspect (db), a concrete value for this function may be that aspecific patient (Patient Xyz) is not present in the database A_(db)^(XyzNotFound). Hence, the aspect for the database may be defined by avariety of functions, such as:

Adb=Σ _(Patients)(A _(db) ^(i))

[0035] with A being defined by the precise makeup of the Patient's dataentries. As such, the definition and refinement of the aspect functionsmay be performed in a myriad of ways.

[0036] The aspects which define the state equation may be orthogonal toone another. Changes in the aspects lead to a change in the state, withthe state of the system being altered by altering the aspectsindividually and independently of one another. This is in contrast to aforced restart of the system whose changes are interdependent and resultin changing parts of the system that are operating correctly along withother parts of the system that require modification. Thus, theseindividual and independent alterations of aspects lead to orthogonaltransitions of the system state. While the aspects are defined asindependent of each other, normal operation of the testing software islikely to restrict possible aspect changes to the current value of otheraspect functions. A medical imaging application, for example, mayinclude aspects which, in practice, are interrelated. For example, theuser privileges aspect may not depend on database aspect (e.g., databasecontent). However, normal operation of the medical imaging applicationsoftware restricts the values of some of the aspects, including thedatabase content which depends on the user privileges. Therefore,aspects in practice may be connected, limiting some of the changes tothe aspects. These changes are discussed in further detail below.

[0037] As discussed above, an aspect may be alterable individually andindependently of other aspects. The design of the application under testmay facilitate individual and independent alterability of the aspects.The application under test may be designed as a series of components,with interfaces for each of the components. Aspects may be assigned tothe components allowing the independent and individual change of each ofthe components via the interfaces. An example of this may be for amedical imaging application which includes a database and a databasescripting interface which allows individual and independent changes tothe database separate from other components in the system. In thismanner, the change in the database aspect amounts to an orthogonaltransition of the system state.

[0038] As shown above, the state equation, M, is defined with trailingspaces. The trailing spaces indicate that there may be further aspectsnot formulated or not known. These further aspects are unconsideredaspects. However, proper definition of aspects may still allow robustaspect based recovery if aspects are changes in aspects are independentor orthogonal to one another. As merely one example, consider thefollowing aspects: security, logged in user, database inventory. Thesecurity and logged in user aspects are defined from a system point ofview and may be accessible via command line interfaces. The databaseinventory aspect is defined as what is visible to the current user inthe Patient Browser subapplication (user point of view). In the instancewhere a dataset is protected, it is not visible to the current user. Ifthe protection for the dataset is removed through a command lineinterface, the transition is orthogonal for logged in user aspect andsecurity aspect. Appendix A lists a tool, which may be written inVBScript or Jscript, for determining the state and recovering the stateof the security aspect. However, in the Patient Browser subapplication,that previously hidden dataset will become visible, so that, for thedatabase inventory aspect, there is a dependency and nonorthogonaltransition.

[0039] One aspect of this embodiment is determining the state of theapplication under test and, if the application under test is not in anexpected state, automatically changing the state of the applicationunder test to the expected state or to another state. This determinationof the state of the system may be performed for any state of the system.Further, the determination of the states may depend on the testingroutine or may be independent of the testing routine. For example, thecheck for the system default state, which is entered after start-up ofthe system, may be independent of the testing routine chosen. Each ofthe aspects for the system default state may comprise a generic check todetermine whether the particular aspect is in the system default state.For example, an aspect object has a fixed part that is set during itsconstruction, which may include the aspect's name, associated systemdefault, and associated recovery. These states which do not depend onthe testing routine may be termed as generic to operation of the system.The generic states allow for comparison of the values of the aspects ofthe system with values of the aspects for the generic states. Othernon-generic states may require checks which are specific to the testingroutine. These non-generic or dynamic states support a dynamic part forthe aspect object that varies according to the test (precondition,precondition verification, and postcondition). The current state of theapplication under test may be determined by examining the aspects, suchas by comparison of the aspects of the current state with aspects of aknown state, such as system default. If it is determined, based on thecomparison, that the current state is not in the expected state, thesystem is in an unknown or undesired state. In that instance, theaspects associated with the state may be changed in order change thestate of the system. The aspect based recovery may be implemented at anystate of operation of the system, from system default, to precondition,to postcondition, as discussed in more detail below. Typically, eachaspect may have a recovery to a known state, such as the system defaultstate. In one embodiment, the aspect based recovery consists ofmodifying aspects of the state of the system to aspects of a known state(such as the system default state). Additional cleansing of the systemmay optionally be implemented.

[0040] The changing of the state allows for a fast and efficientrecovery system. Otherwise, the system under test may remain in theunexpected or unknown state, thereby wasting testing time, orpotentially requiring a restart, reboot or reinstallation to continuethe testing of the system. For example, the present invention differsfrom rebooting, which is a brute force method of recovery, as discussedin the background section. Rebooting may changes the entire system withno control over which portions of the system may be modified. Bycontrast, the aspect based recovery may select which aspects may berecovered or modified. By analogy, rebooting and aspect based recoverymay be likened to cleaning a blackboard. The blackboard may have writingon it in various places. Rebooting washes clean all of the writing onthe blackboard, and then rewrites on sections of the blackboard as partof the rebooting initialization process. By contrast, aspect basedrecovery may first record the writing on the blackboard (to provide thestate of the system), may selectively clean certain parts of theblackboard without wiping the entire blackboard clean, and may rewritethe parts cleaned. In this manner, aspect based recovery may tailor therecovery of the system from an unknown state to a known state withoutcorrecting parts of the system that do not need correcting and withoutunnecessarily wasting time or resources.

[0041] An example of the savings in using aspect based recovery is thatautomated aspect based recovery may require 1 minute per aspect whereasmanual recovery may take as much as 10 times as long. Estimating that30% of all test scripts must be recovered in some way leads to anenormous time savings in using the automated recovery.

[0042] Another embodiment is providing information regarding the systemunder test if the system is not in an expected state. The informationprovided may include data regarding one, some, or all of the aspects forthe unexpected state and may include data regarding one, some, or all ofthe aspects for the expected state.

[0043] Referring to FIG. 3, there is shown a flow chart of operation ofone embodiment of the aspect based recovery system and method. Referringto FIG. 4, there is shown a class diagram for objects coded using objectoriented programming in an aspect based recovery system and method.FIGS. 5-7 show flow diagrams for the object oriented coding in FIG. 4,with FIG. 5 showing a flow diagram with no abnormal behavior detectedwhile executing a test, FIG. 6 showing a flow diagram with abnormalbehavior and recovery during precondition, and FIG. 7 showing a flowdiagram with abnormal behavior and recovery during postcondition.

[0044] As shown at block 68, the application under test is powered on.After power-up, the batch of testing sequences is started, as shown atblock 69. The batch of testing sequences may be manually started by theoperator in order to test the application under test. In preparation forrunning a testing sequence, the values for aspects during the testingsequence are initialized. This is shown in FIG. 5, which shows thescript initialization for Universal Interface Test Automation (UITA) forthe objects shown in FIG. 4. For example, as shown in FIGS. 4 and 5,SetStateDynamics includes values for the aspect name (aspectName_in),the aspect precondition for the test (preCond_in), the basis to verifythat the precondition has been met for a certain aspect(preCondVerify_in), the description of the precondition(preCondDescr_in) and the postcondition for the aspect to return tosystem default (postCond_in). After executing the batch, the applicationunder test enters the system default state, as shown in FIG. 2. Afterinitialization of the system, the system may be checked to determine ifit is in the default state, as shown at block 70.

[0045] Table 2 describes example aspects in a medical imagingapplication (e.g., magnetic resonance, ultrasound, computed tomography,x-ray or other therapeutic or diagnostic imaging), with example valuesfor the system default and the example aspect recovery. TABLE 2 ExampleAspects Aspect Example Aspect System Default Example Aspect Recovery GUINo message boxes or Close all message windows open in either of theboxes and dialog taskcards. windows. No patients loaded in either ofUnload all patients the taskcards. from all taskcards. Viewer taskcardup front. Close any additional message boxes. Activate Viewer taskcardDatabase Local Database empty. Remove all jobs from Exchangeboard (e.g.,the exchangeboard. clipboard) empty. Clear the database Security Onlyusers from Testhospital Reset user table. configured. Reset user grouptable. Only Testhopsital user groups Reset privilege table. configured.Logout current user, Only Testhospital privileges login default user.configured. Default user logged in.

[0046] As shown in Table 2, one aspect is Graphical User Interface(GUI). In the system default state, the GUI should include no messageboxes or windows open in either of the taskcards, no patients loaded ineither of the taskcards, and the viewer taskcard is in the front of thedisplay. As discussed subsequently, the current aspects of the systemare compared with the aspects of the default system. If the aspectsdiffer, the current aspects may be changed to the default aspects. Asshown in Table 2, example changes of the aspects are shown.

[0047] The analysis to determine whether the system is in the defaultstate may depend on: (1) the choice of aspects to be examined; (2) thetype of examination of the aspects chosen; and/or (3) the sequence ofexamination of the aspects. One, some, or all of the aspects may bechosen for examined to determine whether the system is in the defaultstate. In one embodiment, all of the aspects are checked. In analternate embodiment, only more important aspects for the operation ofthe system are examined.

[0048] Further, the type of the examination of an aspect to verifywhether the system is in the default state may take several formsincluding: (1) analysis of an aspect independently of or dependent onother aspects in the system; (2) analysis of an aspect for particulardata; (3) analysis of an aspect for general parameters; (4) analysis todetermine whether a particular event has occurred. Typically, theaspects may be analyzed independent of one another to determine whetherthey are in the default state. Independent examination of aspects may beperformed by using batches, such as from installation routines, COMservers, product component test APIs, etc. For example, if an aspectrelates to a database entry, examination of the database may beperformed independent of other aspects by using an application programinterface (API) which allows direct examination of the database.Appendix B lists a tool, which may be written in VBScript or Jscript,for determining the state of the database aspect. Alternatively, theaspects may be examined in a manner which depends on other aspects inthe system, as discussed in more detail below. The examination of anaspect to verify whether the aspect is in system default may comprise:analyzing the aspect for particular data (such as determining if thespecific patient data was entered into RAM); analyzing the aspect forgeneral parameters (such as determining if the loaded data correspondsto alphanumeric characters); or analyzing the aspect for occurrence of aparticular event (such as if a message box is open). In Table 2, forexample, the GUI, database and security aspects may be analyzed. For thedatabase aspect, the database may be examined to determine whether thedatabase and the exchangeboard are empty.

[0049] If the system is not verified to be in system default, as shownat block 70, the recovery procedure is entered, as shown at block 82.The system is not in system default if the aspects of the current statedo not match the aspects of the default state. This determination may beperformed by comparing the aspects of the current state with the aspectsof the default state. The recovery procedure is discussed in more detailbelow. Alternatively, if the system is verified to be in system default,as shown at block 70, the next test script is accessed, as shown atblock 71. The preconditions are then established for the test script, asshown at block 72. For example, the preconditions are established for atleast some of the aspects of the system, as shown in FIG. 5, which runsthe preconditions to establish the conditions for the test.

[0050] After which, it is determined whether the preconditions wereestablished properly, as shown at block 74. This step may be aprecondition verify step, providing verification that the preconditionswere successfully established. This is shown in FIG. 5 as thePreconditionVerify step. Ordinarily, verification of the preconditionstep is not performed. Thus, prior to executing the test script, onecannot determine if the conditions precedent to the test have beenestablished correctly. Specifically, if the system fails either duringor after the test, one cannot automatically determine under previoustesting automation systems whether the failure was due to theprecondition step or due to the test script step.

[0051] As discussed above, the testing system may determine the state ofthe system at any point during the testing process. This may beperformed, for example, after establishing preconditions to determine ifthe testing system is in script begin state (block 62 in FIG. 2)indicating that the preconditions have been established successfully orthat the system is in a system unknown state (block 64 in FIG. 2)indicating that the preconditions have not been establishedsuccessfully.

[0052] The analysis to verify whether the precondition(s) has been metmay depend on: (1) the choice of aspects to be examined; (2) the type ofexamination of the aspects chosen; and/or (3) the sequence ofexamination of the aspects.

[0053] One, some or all of the aspects may be chosen for examination todetermine the state of the system. In this manner, the automated testingmay determine whether one, some or all aspects have failed to be in theexpected state (the script begin state). For example, only aspects whichrelate to the preconditions may be examined. Specifically, thePreconditionVerify function may examine each aspect of the system whichis affected by the precondition step. As one example, the preconditionstep may require the loading of patient data from a database into RAM.The PreconditionVerify function may examine the database aspect of thesystem which has been affected by the precondition step. As anotherexample, the precondition step may require the loading of patient datain the user interface with certain security protections. In thisinstance, potentially three aspects may be affected by this operation.The first is the database aspect, which requires the loading of thepatient data. To accomplish this, a command line interface may be usedto put the patient data in the database. The second is the userinterface aspect, which includes using a viewing, such as a patientbrowser, to view the data. The third is the security aspect, which wouldapply a protection to the patient data. To accomplish this, a commandline interface may be used to apply a protection to the data.Alternatively, aspects designated as important may be analyzed as well.For example, one or some of the aspects, which may be deemed of greaterimportance to the functionality of the system, may be examined for allprecondition verifications.

[0054] Further, the type of examination of an aspect to verify whetherthe precondition has been met may take several forms. One form is toanalyze the aspect either independently of or dependently on otheraspects in the system. As discussed above, analysis of an aspect may beindependent of other aspects. To analyze the User Interface aspectindependently, for example, an API in Windows® may query whether awindow is open. As another example of independent examination, databasecontent may be accessed by using an API. Typically, if there is embeddedsoftware directed to a certain aspect, examination of that aspect may beperformed independently of other aspects. Alternatively, the database orother aspect may be analyzed in a manner dependent on another aspect,such as the user interface. For example, during the precondition verifystep, the patient browser (which is part of the user-interface) may beopened to determine the contents of the database. The preconditionverify step may analyze, via the patient browser, the contents of thedatabase. Another example discussed above related to a precondition steploading patient data in the user interface with certain securityprotections. The precondition verify step may independently examine, ata system level, (1) whether the database includes the patient data; (2)whether the patient browser; and (3) whether security protections wereattached to the patient data. Alternatively, the precondition verifystep may examine in a dependent manner whether the preconditions havebeen met. One method of dependent examination is by logging in as theuser who has access to the patient data, opening the patient browser anddetermining whether the patient data is accessible to the user. If thepatient data is accessible to the user, the precondition step has beenmet. Another method of dependent examination is by logging in as asecond user who does not have access to the patient data, opening thepatient browser and determining whether the patient data is accessibleto the second user. If the patient data is accessible to the seconduser, the precondition step has not been met. This type of dependentanalysis of an aspect may be desirable if the purpose of theprecondition verify step is not only to determine the contents of thedatabase, but also to determine if the contents are accessible in thespecific application (such as viewing the contents via the userinterface). When analyzing an aspect in a dependent manner and aprecondition is not met for a specific aspect, one may not be able todetermine whether the cause was due to the specific aspect underanalysis or whether the cause was due to examination in a manner whichdepends on another aspect. In one of the examples discussed above, ifthe testing automation software attempts to access the database usingthe patient browser and determines that the data in the database is notprecondition state data, the cause of this may be: (1) that the data inthe database is incorrect (i.e., the specific aspect under examinationis the cause of the error); or (2) that the user-interface hasmalfunctioned so that attempts to access the database through thepatient browser fails (i.e., another aspect which is not the focus ofthis examination is the cause of the error).

[0055] Another form is to analyze the aspect for particular data. Forexample, the database aspect described above may be compared todetermine if the specific patient data was entered into RAM. Anotherform is to analyze the aspect for general parameters. For example, thedatabase aspect described above may be analyzed to determine if theloaded data corresponds to alphanumeric characters. Still another formis to determine whether a particular event has occurred. For example,the graphical user interface (GUI) aspect may be analyzed to determineif a message box is open. Another example of an event is whether acertain user was logged into the system. Still another example of anevent is whether certain data was protected. In many systems, data maybe protected so that only certain login users may access the data. Thisprecondition may be determined by logging in a user that should not haveaccess to the data and determine whether the user may access the data.If the user cannot access the data, the precondition may be verified.Alternatively, if the user can access the data, the precondition is notverified.

[0056] Based on analyzing the aspect(s) of the system, one may determinewhether the precondition has been properly established. If theprecondition has been properly established, the test script is run, asshown at block 76. After the test script is run, a determination is madeas to whether the test script has executed successfully, as shown atblock 77. This determination may comprise determining whether the systemis in the script end state (block 66 in FIG. 2). For example, thecurrent aspects of the system may be compared with the expected aspectsof the script end state. If the current aspects match the expectedaspects, the state of the computer is in the script end state. If not,the application under test may have crashed and the aspect basedrecovery may be implemented. If the precondition has not been properlyestablished, the system is in an unknown state. Recovery to a knownstate, such as the default state, is then performed.

[0057] Prior to changing the state, one, some, or all of the aspects maybe recorded for subsequent analysis, as shown at block 82. For example,only the aspects which have failed to be in the expected state may berecorded. The precondition verify routine may indicate which aspectshave failed. Alternatively, all of the aspects of the system may berecorded in order to provide a complete picture of the state of thesystem at the time of failure. Along with this information, the expectedvalue and/or event for the aspect may be recorded as well. This may beused to compare with the actual value of the aspect in order to diagnosethe cause of the failure better. Moreover, additional information may berecorded including the time of the failure, the current testing sequence(such as the current test script) the previous test script executed,etc. Additional information may also be logged in, such as processorload, RAM usage, etc. In an alternate embodiment, rather than recordinginformation only after a failure to be in the expected state,information (such as aspects, time, processor load, RAM usage, etc.) maybe recorded periodically even in the absence of a failure. For example,the information may be recorded after every testing sequence or duringcertain parts of the testing sequence.

[0058] After recording the information for failure analysis, one, some,or all of the aspects may be modified to place the system in a knownstate, such as the default state, as shown at block 84. As discussedabove, the state of the testing system may be modified to any statebased on changing aspects which define the state. At least one, some, orall of the aspects are changed to change from an unknown state to thedefault state. In one embodiment, all aspects may be examined todetermine whether they are in the default state. If the aspect is not inthe default state, the aspect is changed to its default state value.

[0059] In an alternate embodiment, fewer than all of the aspects may bechanged to their default state. For example, recovery may be limited toa single aspect such as: a routine to kill all of the popup windows; abatch file that purges the user table and fills it with default data,such as syngo® Test Hospital data, as described below; a routine thatpurges the local database and fills it with default data or keeps thedatabase purged; or a routine to reestablish default systemconfiguration. Each recovery may be implemented so it is generic for agiven aspect, regardless of the testing sequence. Alternatively, aspectbased recovery may execute a series of generic routines, with each ofthe generic routines recovering an aspect, one sequentially after theother. As another example, only the aspects which register an error(precondition has not been met) are changed to the default state. Theprecondition routines determine which aspects have not met theirpreconditions. These aspects may then be modified to their defaultstate. For example, as shown in FIGS. 6 and 7, abnormal behavior mightoccur in either SysDefaultVerify( ) or PreconditionVerify( ). This isreported to GenCommander through the RunPreconditions( ) and RunPostconditions( ) return values. Additionally, AspectManager mightbackreference SeBigBos for detailed error logging. Though not disclosedin the figures, this functionality may be supported by introducing anInitialize(SeBigBos: GenCommander) function. Alternatively, in additionto the aspects which register an error, other aspects are examined todetermine whether to modify the aspects to the default state values. Forexample, certain aspects, designated as important aspects, may beexamined every time a precondition error has occurred. Examples ofimportant aspects may include user interface or database aspects. Theexamination of these aspects may include determining whether the aspectis in the default state. This may comprise comparing the aspect valuewith the default state value. If the aspect is already in the defaultstate, the aspect need not be changed to the default state. If theaspect is not in the default state, the aspect may be changed so that itis in the default state. Alternatively, rather than determining whetheran important aspect is in the default state, the testing system maychange all important aspects without this determination so that they arein the default state. As another example, the aspects may beprioritized, with each aspect being assigned a high, medium and lowpriority. If an aspect fails upon establishing a precondition and theaspect is in the high priority category, only high priority aspects areexamined to determine whether they are in the default state.Alternatively, if a failed aspect is in the medium priority category,high and medium priority aspects are examined. Finally, if a failedaspect is in the low priority category, high, medium and low priorityaspects are examined. As one example of this, a cache aspect may beconsidered a low priority. If the cache typically includes a consistentset of data throughout operation of the system (e.g., a set of users forlogging into a system), the contents of the cache may not need to beupdated back to system default. Alternatively, if the cache aspect isconsidered more important, and the default'system value for the cache isnot in the default state, the contents of the cache may be overwritten.

[0060] Further, the sequence of examination and/or changing of aspectsmay be predetermined or may vary. In one embodiment, the sequence ofexamination and recovery of the aspects is predetermined. Specifically,each aspect is examined in a predetermined sequence. If an aspect isunable to be examined and/or recovered, the aspect is examined and/orrecovered at a later point. This is due to the fact that an error in oneaspect may affect examination or recovery of another aspect. When anaspect cannot be examined or recovered to its default state, theexamination or recovery of the aspect may be performed at least a secondtime after other aspect(s) are recovered. For example, all aspects thatare not examinable or recoverable will be examined or recovered in asecond iteration. Those aspects that cannot be recovered in the seconditeration are examined in a third iteration. This process may becontinued until the aspects that are not recoverable remain constantfrom one iteration to the next. In this instance, a more aggressiverecovery is required, such as a reboot or a reinstallation. In addition,recording of the iterative process steps may be performed. As oneexample, a record may be compiled of the attempts made to determine orrecover certain aspects for one, some, or all of the iterations.Further, the Aspect Manager, as disclosed in FIGS. 4-7, may monitor theattempts to determine and record which aspect hindered the correction ofother aspects. This may be accomplished by examining the sequence of thecorrection of the aspects. For example, if aspect A is unable to becorrected in a first iteration, aspect B is the only aspect corrected inthe first iteration, and aspect A is able to be corrected in the seconditeration, the Aspect Manager may report that the correction of aspect Awas dependent on aspect B. Moreover, the Aspect Manager may recordcertain portions of the aspect for later diagnosis. The recording may bedirected to error messages provided by the system.

[0061] One example of this dependence which may require an iterativeaspect based recovery may be with respect to database objects.Attempting to examine or modify a database object may fail if anotheraspect, such as a user interface aspect, has failed. In this instance,the database object is locked due to the failure of the other aspect. Ifan attempt is first made to the database object to change its value andthe attempt fails, other aspects are modified, such as the userinterface aspect. In this manner, the database object may be modifiedsubsequently, after the user interface aspect is changed to the defaultstate.

[0062] Finally, there may be different types of recovery to a knownstate. In one embodiment, there is only one type of recovery of theaspects to a known state. A generic routine for each aspect modifieseach aspect to the system default state. Alternatively, there is morethan one type of change back to system default. As discussed in moredetail below with respect to FIGS. 8a-e, the type of recovery may beeither micro or coarse. These different types of recovery enable thechoice of how a particular aspect is changed to the system defaultstate.

[0063] After the aspect(s) are changed, the state of the testing systemis examined to determine if it is in the system default state, as shownat block 86. This is similar to the testing of the default state asdescribed above with respect to block 70. If the system is in the systemdefault state, the testing system proceeds to access the next testscript, as shown at block 71. Otherwise, additional automatic cleansingmay be performed, as shown at block 88.

[0064] One type of automatic cleansing may be termed cleansingchemotherapy, suggesting that this type of recovery is similar to thetherapy technique to fight cancer. In cancer chemotherapy, after asuccessful treatment campaign in which body scans show no further signof tumors, the patient is subjected to a final chemotherapy in order todestroy any malignant cells that escaped detection. In a similar way,the cleansing chemotherapy step may be performed in order to correct anysleeper cells of misaligned or erroneous unconsidered aspects that couldbe leftover in the system and potentially cause false test results atunpredictable points in time. The cleansing chemotherapy may be appliedas a generic step after every time the aspect based recovery method,which recovers the system back to the default state, is called.Alternatively, the cleansing chemotherapy may be applied only after theaspect based recovery method has failed to return the system back to thedefault state, as shown in FIG. 3. One example of the cleansingchemotherapy may be an automatic reboot of the system in order to beable to run the default system startup checks and the aspect conditioncheck an additional time. In addition to rebooting of the system, batchscripts which are executed once for postinstallation setup may beautomatically executed once more. This may be done during power on asdescribed in FIG. 2.

[0065] Another type of cleansing is an intermediate cleansing. This typeof cleansing is not as exhaustive as the cleansing chemotherapydescribed above. Instead, for an application under test and testautomation technique, it may be useful to automatically conductadditional cleanup steps. As shown in FIG. 3, the automatic cleansing isperformed if the aspect based recovery fails to return the system to aknown state, such as system default. Alternatively, the automaticcleansing, such as intermediate cleansing, may be automaticallyperformed after aspect based recovery whether or the aspect basedrecovery was successful. Further, the automatic cleansing may beperformed before or after the aspect based recovery. One example ofintermediate cleansing may include killing or removing all popupwindows. Another example of intermediate cleansing may include theautomatic ability to terminate a service spawned of the applicationunder test. The application under test may spawn a separate service,such as a virus scan. The intermediate cleansing may terminate the virusscan. This intermediate cleansing step is optional and may not existindependent of other recovery routines. For example, the intermediatecleansing may be part of the individual aspect recoveries of the aspectbased recovery method.

[0066] After performing the automatic cleaning, the state of the testingsystem is examined to determine if it is in the system default state, asshown at block 90. This is similar to the testing of the default stateas described above with respect to block 70. If the system is in thesystem default state, the testing system proceeds to access the nexttest script, as shown at block 71. Otherwise, additional manualcleansing may be performed, as shown at block 92. The manual cleansingmay include reinstallation of the software for the system. For complexsoftware products, however, the reinstallation may prove extremelytime-consuming. Reinstallation may comprise manually setting up theoperating system, manually installing the application software andmanually configuring site specific configurations. For complex softwareproducts, however, the reinstallation may prove extremelytime-consuming.

[0067] As discussed above, if the preconditions have been properlyestablished, the test script is run, as shown at block 76. Then, thepostconditions may be run. Postconditions, as discussed above, may beviewed as a “cleanup” stage for cleaning up the system prior toexecuting the next test. In one embodiment, the postconditions returnthe system to the default state, as shown at block 78. Alternatively,postconditions may bring the aspects back to a state different fromsystem default. For example, postconditions may alter the state of thesystem to a test default state, which is the default state for a groupof test cases. The test default state may be different from the systemdefault state. Returning the system to a test default state, rather thansystem default state is acceptable if the testing sequences begin fromthe test default state. However, if different testing sequences beginfrom different states, additional cleanup may be required. In still analternate embodiment, postconditions need not be implemented. Rather,the aspect based recovery mechanism may be executed immediately afterrunning the test script and bring the system back to system default.

[0068] After postconditions are run, the system may be checked todetermine if it is in the system default state, as shown at block 80. Ifyes, the next test script is accessed, as shown at block 71, and theprocess is repeated. If the system is not in system default, the aspectbased recovery may be executed, as discussed above.

[0069] One example of application of the aspect based recovery may beautomated testing of a medical imaging application, such as syngo®.syngo® is a comprehensive software solution for medical imaging tasks,applications and systems. syngo® enables all workplaces, fromacquisition modality to independent workstations, to share the sameintuitive user interface and operating principles. syngo® provides thisuniform working environment throughout clinical networks and beyond.syngo® is designed to integrate medical imaging seamlessly into thecomplete clinical workflow from registration to billing.

[0070] To accomplish this integration of medical imaging, syngo hasarchiving and networking functionality including: (1) communicatingpatient and examination data; (2) exporting/importing image data to/fromMOD, CD-R or floppy; (3) sending/receiving image data to/from otherDICOM (Digital Imaging and Communications in Medicine) nodes in thenetwork; and (4) searching and retrieving image data in the network.

[0071] syngo® enables the exchange of information between systems fromdifferent vendors through networks or media. This leads to improvedefficiency in diagnosis and therapy. syngo® integrates medical imaginginto a complete clinical workflow from referral to billing and therebyenables an optimized utilization of imaging systems. syngo® supports acomprehensive set of DICOM objects including: Computed Radiography (CR);Computed Tomography (CT); Digital X-Ray (DX); Mammography (MG); MagneticResonance (MR); Nuclear Medicine (NM); Positron Emission Tomography(PET); Radiation Therapy (RT); Secondary Capture (SC); Ultrasound (US);US multi-frame; Waveforms; X-ray Angiography (XA); XA multi-frame; andX-ray Radiofluoroscopy (XRF). Further, syngo® provides a comprehensiveset of DICOM functions, compliant with IHE and VA regulations including:DICOM Worklist to register incoming patients directly from thescheduling system (e.g., the Radiology Information System); DICOMModality Performed Procedure Step to enable complete procedure trackingfor billing and utilization management; DICOM Print to expose images ona wide range of networked cameras and paper printers; DICOM Storage ofinformation on connected nodes; DICOM Storage on a wide range ofexchange media (e.g., Magneto-optical disc MOD, CD-R, Floppy); and DICOMQuery & Retrieve for searching and retrieving patient data andexamination data in the network. DICOM Storage Commitment.

[0072] Typically, the workflow progresses from registering a patient,acquiring an image for the patient, viewing the image, post processingof the image, and filming. syngo® may accomplish these tasks for allsegments of medical imaging including, for example: patientregistration; patient browsing; viewing; filming; 3D; fly-through; andimage fusion.

[0073] Patient registration may include: all relevant patientdemographic and examination data; direct entry or retrieval from a DICOMworklist provider; direct launch of the acquisition with a singlemouse-click; procedure tracking with DICOM MPPS (Modality PerformedProcedure Step); and correction of wrong or missing entries in patientdemographics and DICOM attributes.

[0074] The patient browser may include consistent access from allapplications on the system to patient and examination data. Viewing mayinclude the functionality of: loading images from the Patient Browser ordirectly from other syngo applications (e.g., examination or 3D postprocessing); displaying and arranging the images in the way best suitedto the diagnostic problem; reviewing, processing and evaluating theresults and prepare them for diagnostics; and sending the images tofilming, storing them or sending them to other locations in thehospital.

[0075] Filming may include the functionality of previewing on a 1:1basis of a virtual film sheet; directly manipulating imagesinteractively; exposing (manually or automatically) on film or paper;controlling film job status (repeat, cancel, re-direct). 3D (Processvolume data sets from various modalities; Merge overlapping 3D-seriesinto a single consistent 3D series; Confine the data sets with clipboxor irregular volume-of-interest; Display the data as high-quality;Multiplanar reconstruction (MPR); Shaded surface display (SSD); Maximumintensity projection (MIP)); Volume Rendering Technique and ObjectEditor). CorRea relates to additional data associated with an imagewhich provides an identification of the image for billing, storing, andanalysis purposes. The additional data may include the patientidentification, patient date of birth and sex of patient. In thismanner, a specific image may be reference using the additional data,such as the patient identification. CorRea is a data editing tool,allowing for modification of the additional data associated with theimage.

[0076] As discussed above, syngo® integrates medical imaging into acomplete clinical workflow from referral to billing. MPPS (modalityperformed procedure step) allows for integration of these separatehospital applications, from billing, patient records, purchasing, etc.For example, if a patient is analyzed using a CT scanner, the specificimage of the patient and the X-ray dosage resulting from the scan may besent to the patient records application. Further, the informationregarding the scan may be sent to the billing application. Finally,materials used from the CT scan may be reported to the purchasingapplication.

[0077] Example aspects for testing automation of the syngo® applicationare shown in FIGS. 8a-e. As shown in the figures, the aspects aregrouped by state group, which categories common aspects. For example,the database state group comprises aspects default data, testcasespecific data, and exchangeboard. Further, a description is provided foreach aspect, the system default for each aspect, and possible statesafter a testing procedure is executed. Aspect based recovery mayimplement different types of recovery to the default state. For example,as shown in FIGS. 8a-e, different types of recovery may be micro cleanupand coarse cleanup. The micro cleanup is less invasive than the coarsecleanup. Implementation of aspect based recovery may be solely usingmicro cleanup, solely using coarse cleanup, or a combination of microand coarse cleanup. For micro cleanup, the aspect based recovery maymodify aspects such as by reloading or removing data, closing windows,etc. For coarse cleanup, the aspect based recovery may purge andreestablish default configuration or may restart a component of thesystem. The forced restart of the component is contrasted with a reboot,which is a forced restart of the entire application under test. Asdiscussed above, one benefit of the aspect based recovery is theindividual correction of aspects. Designing an application under testwith components enables better testing and control of aspects which areassigned to the components.

[0078] As shown in FIGS. 8a-e, some of the state groups are specific tothe syngo® application including: patient registration; patient browser;viewer; filming; 3D; CorRea; and MPPS. When subjecting an application,such as syngo to automated testing, aspects of syngo may be defined, asdisclosed in FIGS. 8a-e. The automated testing may execute test scriptsfor each of the functionalities, in a variety of ways, of registering apatient, acquiring an image for the patient, viewing the image, postprocessing of the image, and filming.

[0079] While this invention has been shown and described in connectionwith the preferred embodiments, it is apparent that certain changes andmodifications in addition to those mentioned above may be made from thebasic features of this invention. Aspect based recovery may be appliedto a variety of products, such as software based, hardware based, orsoftware/hardware based products. Moreover, aspect based recovery may beused to determine the state of the product at any point of its operationlife, from automated testing of the product to operation of the productin the field. Further, aspect based recovery may modify the state of theproduct, such as from an unknown state to a known state (e.g., defaultstate).

[0080] In addition, there are many different types of computer softwareand hardware that may be utilized in practicing the invention, and theinvention is not limited to the examples described above. The inventionwas described with reference to acts and symbolic representations ofoperations that are performed by one or more electronic devices. Assuch, it will be understood that such acts and operations, include themanipulation by the processing unit of the electronic device ofelectrical signals representing data in a structured form. Thismanipulation transforms the data or maintains it at locations in thememory system of the electronic device, which reconfigures or otherwisealters the operation of the electronic device in a manner wellunderstood by those skilled in the art. The data structures where datais maintained are physical locations of the memory that have particularproperties defined by the format of the data. While the invention isdescribed in the foregoing context, it is not meant to be limiting, asthose of skill in the art will appreciate that the acts and operationsdescribed may also be implemented in hardware. Accordingly, it is theintention of the Applicants to protect all variations and modificationwithin the valid scope of the present invention. It is intended that theinvention be defined by the following claims, including all equivalents.Usage: seci login <username> <pwd> <domain> seci logoff <userId> secigetCurrentUser seci authorize <userId> <application> <token> secidbAuthorize <userId> <loids>... seci checkAccess <userId> <application><token> <requiredAccessMask> seci audit <applName> <funcPointId>[<parameter>...] seci secCmd ... seci auditCmd ... seci dbCmd ... secireconfigure seci exitServer seci protectionList seci getList <tokenName>seci readAllTokens <userId> seci userConfig <path> <key> secigetDefaultProt <userId> seci getEffDefaultProt <userOrGroupSpec> secisetDefaultProt (install|uninstall|read|save) <userOrGroupSpec><protectionList> seci setObjProtections (<loid>|<securityId>)<protectionList> seci getObjProtections (<loid>|<securityId>) ... seciconfigValue install <categoryName> <paraName> <value> [<Comment> <type><options> <listVal1> ... ] seci configValue (uninstall|read)<categoryName> <paraName> seci configValue write|append <categoryName><paraName> <value> seci privilege (install|uninstall|read) <applName><funcToken> [<comment>] seci auditConfig (install|uninstall|read)<applName> <funcPointID> [<comment>] seci setPermissions <applName><funcToken> (clear|<userOrGroupSpec> (<allowString>|-) <denyString>)seci getPermissions <applName> <funcToken> <userOrGroupSpec> secisetAllApplTokenPerms <userOrGroupSpec> (<allowString>|-) <denyString>)seci directPermission (<userName> <domain><allowString>|<directPermString>) seci copyToken <origAppl> <origToken>[<newAppl> <newToken>] seci certificate (help|<options>) secisetFileSecurity <filename> <userSid> (N|R|W|F) seci getLocalizedAccountEveryone Used environment variables:SECICMD_TESTLOOPNUM=<numberOfExecutions>  SECICMD_TESTLOOPSLEEP=<sleepTimeInMillisecs>

[0081] How to use the txtExplorer 1: Operation modes The txtExplorer canbe used in three different modes: a- Commandline mode All Commands thatare given in the CommandLine will be executed in their Order. Everycommand must be preceded by a “-”. Example:     txtexplorer -dmydatabase -listpatients   will invoke the Tool, open the database“mydatabase” and list all Patients in that Database. b- Interactive modeWith the Commandline Argument “-i”, the Tool switches to the Interactivemode. Here, all commands are entered in a shell-like environment. No “-”must be entered before any command. Example:     txtexplorer -dmydatbase -i   will invoke the Tool, open the database “mydatabase” andswitch into interactive mode.     >> listpatients   will list allpatients. c- File mode In the file mode all commands are read from afile. The Syntax is the same as in interactive mode. Example:    txtexplorer -file myfile   will invoke the Tool, switch to File modeand executer all commands in the file “myfile” This is common for allmodes: - Commands and Arguments must be separated by one or moreSpaces. - Commands are not case-sensitive - Commands may be abbreviated,but one should be aware of ambiguities. - If an Argument containsSpaces, it must be Quoted (Example: “Argument with Spaces”). - ListExpansion: If a command requires a Loid as an argument, one can alsoenter the List Index of the previously executed Listxx command. - A “#”denotes a comment till the end of the Line. - Attribute Lists:Attributes in () are not accessible by the generic Interface, so theyare not modifyable. 2: Example Invoke the Tool, open Database <dbname>and go into interactive mode:   txtExplorer -d <dbname> -i List allavailable commands:   help List all Patients in the Database:  listpatients Select the First patient in the List and dump the wholeTree:   list 1 Switch mode: Dump only Attributes with a value:   nullsoff Dump all Patients and its Trees:   listpatients   forall %pat { list%pat } Show all CsaStoreDB Locations   liststoredb Create a newCsaStoreDB Location at c:\TEMP   createstoredb mystoredb c:\TEMP Load anImage into the Database   load c:\myimages\ct01.img Unload an Image byLoid and save it in d:\newimage.img   unload 123.0.99223 d:\newimage.imgGet All Studies for a Patient   liststudies 123.0.99220 Get All Seriesfor Study No. 1 in the previous displayed List created with liststudies  listseries 1 Get All Images for Series No. 1 in the previouslydisplayed List created with Liststudies   listimages 1 3: Extending 1-Create own DLL by deriving from class pxExplorerServer. 2- Create‘Factory’ function createInstance. 3- Tell Explorer which DLL to load by  a: commandline : txtExplorerd @myDLL -d ......   b: Env Var: SETDLL_PXEXPLORER=mydll.dll or SET DLL_PXEXPLORERD=mydebugdll.dll ? :Help - List all commands help : Help - List all commands session : SetSession mode to pessimistic (=default) or Optimistic. Must be usedbefore opening the first database ! open : same as -d d : Open Databasedbname medcom : Open all Databases defined in MedCom COnfigurationsetdefaultDB : Set Database <arg0> as Default Database setdefaultLo :Set Lock <arg0> as Default Lock. Possible are NOLOCK, RLOCK, WLOCK(RLOCK is default). getdefaultDB : Get current Default Database listDB :List all open Databases close : Close Database <arg0> commit : CommitChanges (this is the default when leaving the Tool) zapcods : CommitChanges and clean up COD-Table. rollback : Rollback changes list : listall Instances that are in the Closure of Loid <arg0> data : ListInstances and data (on=default) or list only Instances (off) data on(default) : List Classes and Attributes data off  : List only Classesrecurse : Access recursively the whole closure (on=default) or not (off)listPatients : list all CsaPatient Instances listStudies : list allCsaStudy Instances for the given PatientLoid <arg0> listSeries : listall CsaSeries Instances for the given CsaStudyLoid <arg0> listImages :list all CsaImage Instances for the given CsaSeriesLoid <arg0> check :Perform a Dicom-Sematics-check on Instance <arg0> (and closure) delete :Delete Instance with Loid <arg0> and its Sub-Tree in the DB forcedelete: (Forced) Delete Instance with Loid <arg0> and its Sub-Tree in the DBcreateInstan : Create a Instance of class <arg0> in the DB saveAsNew :duplicate Document <arg0> as a new one (modif Uids). Needs ExchangeBoardcopyDocument : duplicate Document <arg0> into the ExchangeBoard move :move Document <arg0> to another database <arg1>. Links to old DB willremain ! Needs ExchangeBoard link : link Document <arg0> to RootDocument <arg1>. unlink : unlink Document <arg0> from its Root Document.query : Run a Database query using the Versant Query Language, f.e.query “select selfoid from CsaPatient”. lock : Lock Object with loid<arg0> with Lock <arg1> (RLOCK, WLOCK, ULOCK, ...) nulls : List used andempty Attributes (on=default) or not (off) nulls on (default) : Listalso empty Attributes. nulls off : Do not List empty Attributes. modify: Modify Instance with loid<arg0>; set Attribute <arg1> , Index <arg2>with Operation <arg3>   to value <arg4>. Operations are:Set/Remove/Add/RemoveOne/InsertOne/ReplaceOne.   All Arguments 4 must begiven, even if they don't make sense for the specified Operation. file :Read commands from file <arg0> (same Syntax as in -i mode) i : Switch tointeractive mode (commands are the same, but omit leading ‘-’ !) logging: Print CsaStatus Items in Errormessage (on=default) or not (off) exit :Exit interactive mode / File mode quit : Same as exit server : Servermode: Await commands in File <arg0>, delete file after execution. Writeresults in file <arg1> pad : Shorten data to fit on 120-chars-Line (on)or not (off=default) load : Load an(more) Image given by <arg0> into theDatabase. Wildcards *ARE* Supported. unload : UnloadImage/Series/Study/Patient with Loid <arg0> into Dicom Message in file<arg1> listStoreDb : List all CsaStoreDB Instances in the DBcreateStoreD : Create a CsaStoreDB Instance with Name <arg0> and Path<arg1> in the DB autosave : Change autosave Flag (commit after eachcommand)   autosave on (default): Every command call commit afterexecution autosave off (default): commit/rollback must be called by theuser. ForAll : Repeat following Block for all listed Loids; Syntax:forall xx { cmds xx cmds } Repeat : Repeat following Block( = <arg1>)exactly <arg0> times (−1 means forever);  Syntax: repeat num { cmds cmds} loadlibrary : Load a library, whose name is given in <arg0> (DLL)CAUTION: Will only work BEFORE opening a Database Session ! initDbInfo :Initialize dbInfo Object in default Db and set with current values.getDbInfo : Get DbInfo Object in default Database and list its contentscheckConfig : Check Versant/Database configuration sleep : Sleep someseconds (arg1) date : Print Date and Time echo : Print a string <arg0>pxsv : Send Cmd <arg0> and data <arg1> to PatexServer xmldump : Dumpdata into XML. Arg0 = type: COMPOSITE, CLOSURE, ENTITY, QUERY, Arg1 =Loid or Classname, Arg2 = filename sort : Sort <arg0> according criteria<arg1>. Always force a resort. getsorted : Get sorted list for object<arg0> according criteria <arg1> updatescs : Check and update SCSattribute for loid <arg0>.

1. Method for executing an automatic recovery routine for a computersystem, the computer system having a plurality of states defined byaspects, the method comprising: providing a software routine for placingthe computer system in an expected state defined by an expected aspect;executing the software routine on the computer system after which thecomputer system is in a current state defined by a current aspect;automatically determining whether the current aspect matches theexpected aspect; automatically determining whether the current state isthe expected state based, at least in part, on automatically determiningwhether the current aspect matches the expected aspect; and executing anautomatic recovery routine if the current state is not the expectedstate, the automatic recovery routine modifying the current aspect. 2.The method of claim 1, wherein the software routine comprises anautomated testing routine for the computer system.
 3. The method ofclaim 1, wherein executing an automatic recovery routine changes thestate of the computer system to a system default state.
 4. The method ofclaim 3, wherein the expected state comprises the system default statedefined by a system default aspect; and wherein modifying the currentaspect comprises modifying the current aspect to the system defaultaspect.
 5. The method of claim 1, wherein the software routine comprisesan automated test precondition routine which places the computer systemin a precondition state defined by a precondition aspect, and whereinautomatically determining whether the current state is the expectedstate comprises determining whether the computer is in the preconditionstate after execution of the automated test precondition routine.
 6. Themethod of claim 5, wherein modifying the current aspect comprisesmodifying the current aspect to a system default aspect.
 7. The methodof claim 1, wherein the aspects are selected from the group consistingof graphical user interface, database, security, user configuration,site configuration, and current user.
 8. The method of claim 1, furthercomprising recording the current aspect.
 9. The method of claim 1,wherein an aspect comprises a graphical user interface aspect, whereinmodifying the current aspect comprises closing all popup windows. 10.The method of claim 1, wherein the expected state is defined by aplurality of expected aspects; wherein the current state is defined by aplurality of current aspects; wherein automatically determining whetherthe current state is the expected state comprises comparing the currentaspects with the expected aspects; and wherein executing an automaticrecovery routine comprises selecting which aspects are to be modifiedand modifying the selected aspects.
 11. The method of claim 10, furthercomprising determining which current aspects do not match the expectedaspects; and wherein selecting which aspects to be modified comprisesselecting the current aspects which do not match the expected aspects.12. The method of claim 11, wherein executing an automatic recoveryroutine comprises: attempting to modify the first aspect to the expectedaspect; determining that the attempt to modify the first aspect failed;modifying other aspects to their expected aspects; and attempting for atleast a second time to modify the first aspect to the expected aspect.13. The method of claim 12, wherein determining that the attempt tomodify the first aspect failed comprises comparing the first aspect withthe expected aspect.
 14. The method of claim 11, wherein the automaticrecovery routine comprises iteratively attempting to modify aspects, theautomatic recovery routine attempting to recover aspects until a numberof aspects that are unrecoverable from one iteration to a subsequentiteration remains constant.
 15. The method of claim 1, wherein modifyingthe current aspect comprises modifying the current aspect independent ofchanges to other aspects.
 16. The method of claim 1, further comprisinga forced restart of the computer system.
 17. Method for executing arecovery routine for a computer system, the computer system having aplurality of states defined by aspects, the method comprising: executinga software routine; automatically determining, after executing thesoftware routine, whether the computer system is in an unknown state;and automatically changing from the unknown state to a known state ofthe computer system, the step of automatically changing consisting ofautomatically modifying aspects of the computer system to place thecomputer system in the known state.
 18. The method of claim 17, whereinautomatically changing from an unknown state to a known state compriseschanging from an unknown state to a system default state.
 19. The methodof claim 18, wherein the expected state comprises the system defaultstate defined by system default aspects; and wherein modifying aspectscomprises modifying at least some of the aspects to their system defaultaspects.
 20. The method of claim 17, wherein automatically determining,after executing the software routine, whether the computer system is inan unknown state comprises examining at least some of the aspects of thecomputer system.
 21. The method of claim 20, wherein the softwareroutine is for placing the computer system in an expected state definedby an expected aspects; wherein after executing the software routine,the computer system is in a current state with current aspects; whereinexamining at least some of the aspects of the computer system comprisesexamining the current aspects with the expected aspects to determine thecurrent aspects which differ from the expected aspects; and whereinautomatically modifying aspects of the computer system comprisesautomatically modifying the current aspects which differ from theexpected aspects.
 22. The method of claim 17, further comprisingrecording aspects of the computer system, the step of recording beingperformed after automatically determining whether the computer system isin an unknown state.
 23. The method of claim 17, further comprising aforced restart of the computer system, the forced restart beingperformed after automatically changing from an unknown state to a knownstate.
 24. Method for executing a recovery routine for a computersystem, the computer system having a plurality of states defined byaspects, the method comprising: providing a software routine for placingthe computer system in an expected state defined by expected aspects;executing the software routine on the computer system after which thecomputer system is in a current state defined by current aspects;automatically analyzing the current aspects of the current state of thecomputer system with respect to the expected aspects of the expectedstate; automatically determining the current aspects which differ fromthe expected aspects and the current aspects which match the expectedaspects; and automatically modifying the state of the computer system byautomatically modifying the current aspects which differ from theexpected aspects and not modifying the current aspects which match theexpected aspects.
 25. The method of claim 24, wherein the expected stateis system default.
 26. The method of claim 24, wherein the softwareroutine comprises an automated testing routine.
 27. The method of claim26, wherein the automated testing routine tests medical imagingsoftware.
 28. Method for executing a recovery routine for a computersystem, the computer system having a plurality of states defined byaspects, the method comprising: providing a software routine for placingthe computer system in an expected state defined by expected aspects;executing the software routine on the computer system after which thecomputer system is in a current state defined by current aspects;automatically analyzing the current aspects of the current state of thecomputer system with respect to the expected aspects of the expectedstate; automatically determining the current aspects which differ fromthe expected aspects and the current aspects which match the expectedaspects; and automatically modifying at least some of the currentaspects which differ from the expected aspects without restarting thecomputer system.
 29. The method of claim 28, further comprising a forcedrestart of the computer system, the forced restart being performed aftermodifying at least some of the current aspects.
 30. The method of claim28, wherein the expected state comprises system default.
 31. Method forrecording aspects for a computer system, the computer system having aplurality of states defined by the aspects, the method comprising:providing a software routine for placing the computer system in anexpected state defined by expected aspects; executing the softwareroutine on the computer system after which the computer system is in acurrent state defined by current aspects; automatically analyzing thecurrent aspects of the current state of the computer system with respectto the expected aspects of the expected state; automatically determiningthe current aspects which differ from the expected aspects and thecurrent aspects which match the expected aspects; and recording thecurrent aspects which differ from the expected aspects.
 32. The methodof claim 31, further comprising: recording the current aspects whichmatch the expected aspects.
 33. The method of claim 31, furthercomprising: recording a time of execution of the software routine. 34.The method of claim 31, further comprising: recording the expectedaspects which differ from the current aspects.
 35. The method of claim34, further comprising: recording the expected aspects which match fromthe current aspects.
 36. An aspect based recovery system for anapplication software on a computer system comprising; a computer systemhaving stored thereon the application software; programming code on saidcomputer system for executing at least a portion of the applicationsoftware; programming code on said computer system for determiningwhether the computer system is in an unknown state; and programming codeon said computer system for changing from the unknown state to a knownstate of the computer system, the programming code for modifying aspectsof the computer system independently of one another to place thecomputer system in the known state.
 37. The aspect based recovery systemof claim 36, wherein the programming code for changing from the unknownstate to a known state consists of programming code for modifyingaspects of the computer system independently of one another to place thecomputer system in the known state
 38. The aspect based recovery systemof claim 36, wherein the programming code for determining whether thecomputer system is in an unknown state comprises programming code forexamining at least some of the aspects of the computer system.
 39. Theaspect based recovery system of claim 38, wherein the programming codefor examining at least some of the aspects of the computer systemcomprises programming code for comparing at least some of the aspects ofthe computer system with aspects of the known state.
 40. The aspectbased recovery system of claim 36, further comprising: programming codeon said computer system for recording at least some of the aspects ofthe computer system.